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A method is presented for calculating the probability of error for 
a digital signal contaminated by intersymbol interference and additive 
Gaussian noise. The method constructs a close approximation to the 
probability density function of the intersymbol interference, circum- 
venting the heretofore formidable computational problems by decom- 
posing the calculation into a sequence of simple calculations. This 
method is applicable to binary transmissions, as well as 4-, 8-, 16-, . . . 
level transmission. The method is rapid enough for use on time-sharing 
facilities. As part of the method, a new and rather simple scheme is 
presented for including the effects of partial response source coding. 
Several interesting examples which use and give insight into the 
method are included. 

I. INTRODUCTION 

In a large class of digital transmission systems, a succession of 
amplitude modulated pulses is sent over a channel to the receiver. The 
signal suffers two main types of distortion: additive noise and inter- 
symbol interference (ISII. The latter arises from dispersion in the 
channel, and is a noise-like process due to the overlapping of many 
neighboring pulses. 1 The number of these neighboring pulses, or "in- 
terferers," depends on the system impulse response, and can be rather 
large for systems with restricted bandwidth. 

To properly analyze a system one must compute its probability of 
error due to the noise and intersymbol interference. The effect of the 
additive noise is simple to analyze, but that of the ISI is extremely 
difficult to find due to the complicated nature of its probability density 
function (pdf). The complexity of the pdf grows exponentially with 
the number of interferers, and for more than about 12 interferers the 
computation of error probability has long seemed impossible. 

2055 
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Until recently researchers have analyzed such systems by one of 
three main routes: (i) they have used more tractable— although less 
meaningful— performance measures such as peak distortion 2 or mean 
squared error, 1 ' 3 (11) they have found upper bounds on the probability 
of error, 4 or (Hi) they have found the probability of error using a 
truncated impulse response, for which only a few interferers were 
assumed to be significant/ 1 ' 6 Unfortunately, in many restricted-band- 
width systems as many as 50 or 60 interferers can be important, so the 
use of a truncated impulse response is inadequate for a large class of 
systems. The goal is therefore to find methods which will treat large 
numbers of interferers while avoiding the exponential growth of com- 
putational complexity. This paper describes such a method. 

Recently two other schemes have been presented which also provide 
accurate estimates of the error probability for large numbers of inter- 
ferers. 7 ' 8 These methods do not provide the pdf of ISI, but calculate 
instead the terms in a series expansion of the expression for the error 
probability. They were devised primarily for the binary transmission 
case, and the extension to multilevel transmission would be rather 
awkward. 

The new technique presented here constructs a close approximation 
to the pdf of intersymbol interference, and uses it to find the error 
probability. Knowledge of the pdf can provide useful information 
about the intersymbol interference process, illuminating for the sys- 
tem designer the relationship between certain types of channel charac- 
teristics and error performance. The technique is applicable to binary 
transmission, as well as to 4-, 8-, 16-, • ■ ■ level transmission. The 
method also allows simple inclusion of certain kinds of source coding 
such as partial response." The ability to treat the multilevel case and 
different kinds of source coding is very important. In feasibility studies 
a system designer might know the general type of channel charac- 
teristics the signal will encounter, but he does not know how sensitive 
the signal will be to various combinations of numbers of source levels 
and source coding. The method given here is envisioned as providing 
a valuable tool in such studies. 

In Section II the probability of error is related to a single random 
variable, the error voltage z, which then characterizes system per- 
formance. It is this random variable whose pdf is sought. Examples 
taken from the applicable class of source coders are presented and it 
is shown that the effects of coding can be transferred from the source 
statistics to the channel description. Alternative performance measures 
are noted for later use. In Section III the details of the method are 
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described with the computer programmer in mind, and various em- 
pirical rules-of-thumb are suggested. In Section IV several simple 
examples are presented which illuminate some of the potential uses of 
the method. In Section V the accuracy of the method is considered, 
and guidelines for choosing some parameter values in the method are 
given. The method presented here is an extended and refined version 
of a technique developed by W. E. Norris. 10 

II. ANALYSIS 

Figure 1 indicates the system under consideration. The source emits 
a symbol every T seconds. The symbols are statistically independent, 
and form the sequence {a k } where each a k takes on one of the A' 
equally likely values m(i) 

m{i) = (2i - N - l)V/(N - 1), i = 1,2, ■■■ , N. (1) 

The outermost levels are ±V volts. The sequence is then passed 
through a coder to form the sequence {c k }. Only two kinds of coding 
are treated in the main body of the paper, although the method is 
extended to more general coders in the Appendix. The two special 
types treated here are the uncoded case and the class IV partial re- 
sponse case," where 

uncoded: c k = a k , 

partial response IV: c k = a k — a k - 2 . (2) 

Partial response IV coding provides useful shaping of the signal spec- 
trum, and precoding of the source sequence can be used to prevent 
error propagation. 9 It is assumed that the sequence {a k } has already 
been precoded appropriately. 

The sequence {c k } modulates the impulse response r(t) of the chan- 
nel. White Gaussian noise is added, and the received signal y{t) is 
sampled every T seconds. The decision device (typically an A/D con- 
verter), determines which of the possible transmitted levels each 
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sample is nearest, and outputs the sequence {d k }. (The receiver can 
base its decision simply on the present sample because of the pre- 
ceding.") Ideally, each d k = c k , but because of the noise and ISI they 
will occasionally differ, and an error is made. The method discussed 
here calculates the probability of occurence of such an event. 
The sample at t = has the form 

2/(0) = £ c k r(~kT) + n, (3) 

where n is a realization of a zero mean Gaussian random variable 
with variance a 2 . Because the sequence {a k } and the noise are sta- 
tionary in a statistical sense, studying the error probability for the 
single sample y(Q) is equivalent to studying it for the entire ensemble 
of samples y(mT). 

Ideally y(0) will equal c . The error voltage z is therefore given 

by 

00 

z = 2 c k x k + n, (4) 

l:=-x 

where 

x k = r(-kT) - 8 k (5)* 

are the "error samples" of the system. It is very convenient to in- 
corporate the effects of any source coding in the error samples. In the 
case of partial response coding: 

QO 

z = 2 (a* — a, k - 2 )x k + n 

i = -oo 
oo 

= £ a k e k + n, (6) 

t = -ao 

where 

e k = x k — x (k +2) (7) 

are the "coded error samples." Thus z of (6) depends on the simple 
statistics of the uncoded source symbols a k . (If no coding is used, then 

e k = x k .) 

The error voltage z is the sum of a large number of independent ran- 
dom variables. Each a k has the same discrete probability density 
function: A set of N equidistant Dirac delta-functions with weights 



* S k = Kronecker delta function : 5„ = 1 ; S k = 0, k ^ 0. 
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1/N. Each a h - is scaled by the coded error sample e k . The n is Gaussian, 
and assumed independent of the a k 's. The pdf p-(-) of z is found by- 
convolving the individual density functions of the constituent random 
variables. Because each of these constituents has a symmetrical pdf 
about z = 0, pA • ) is also symmetrical. Once p.( • ) has been found, the 
error probability for the sample y(0) is simply the probability that 
y(0) is further from c, than from some other level. When c n is an inner 
level this is just the probability that \z\ > V/(N — 1). When Co is an 
outer level only one polarity of z causes an error, and an error occurs 
with the probability that z > V/ (N - 1 ) . For uncoded symbols outer 
levels occur with probability 1/N, while for partial response IV coding 
they occur with probability 1/iV 2 . Hence the error probability is 

P e = 2(1 - AT") f Pt (e) de, (8) 

where h = 1 for an uncoded source, and h = 2 for a partial response 
IV coded source. 

Clearly, the real task in finding P v lies in computing p z (-), since so 
many random variables are involved. The method used to achieve this 
is described in the next section, after two simpler measures of system 
distortion have been introduced. These measures have been widely used 
in the past, and will provide useful definitions in the discussion to 
follow. They are both defined in the noise-free case (i.e., n = 0) here 
in order to isolate the effect of the intersymbol interference. 

2.1 Mean Squared Error 1 ' 3 

The variance of the error voltage z is given by 

E[y(0) - Co] 2 = E[Z W = E[a 2 ] £ el, (9) 

k k 

where E[a-] is the variance of the source. A simple calculation based 
on equation (1) shows that E\a 2 ] = IV-(N + 1)/JV - 1). A nor- 
malized version of the mean squared error then depends only on the 
coded error samples, and will be called MSE: 

MSE = £ el (10) 

k 

2.2 Peak Distortion 2 

The maximum value that z of equation (6) can have when n = is 

max 2 = V 2 I e * r I > (11) 
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which occurs when all of the a k symbols have maximum amplitude V 
and such signs that the interfering symbols add constructively. Such 
a sequence occurs with very low probability. Peak distortion here is 
defined as D: 

D = £ | e k | . (12) 



III. THE COMPUTATIONAL METHOD 

The starting point for computing p s (-) is the set of coded error 
samples {e k }. The only other relevant parameters are V, N and a 2 . 
Because each random variable a* has the same pdf, and this function 
is symmetrical, one need only consider the magnitudes \e k \, and these 
can be relabelled to rank order the samples for convenience. 

3.1 Obtaining the Error Samples 

Channel characteristics are typically measured or calculated in the 
frequency domain, although in some simple cases an analytical expres- 
sion for the channel impulse response is available. If the impulse 
response is indeed available, it is simply sampled as in equation (5) 
to obtain the error samples. Otherwise the error samples are com- 
puted from R(f) as follows.* Because by inspection 

r(kT) = r R(j)e i2 * !kT dj = f" e^ !kT £ R(f - m/T) dj (13) 

the Fourier coefficients of $(/) - £) m R(f - m/T) are desired. A 
Discrete Fourier Transform (DFT)— perhaps using a Fast Fourier 
Transform algorithm— of N s samples of R(f)/N,T, /, = (i - 1)/N.T, 
i = 1, 2, • • • , N B , yields N„ samples of the aliased version t(t) = 
£ m r(t - mT), t k = (k - \)T, k = 1, 2, • • • , N. ." Although the 
samples of f(t) are not identical to those of r(t), for well-behaved channels 
and sufficiently large N, the two are indistinguishable. 

The error samples of equation (5) are formed by removing unity 
from r . The coded error samples (for the partial response IV case) 
are then formed by subtracting from each sample the value of the 
error sample two places to the right in accordance with (7). The last 
two error samples, e iV .-i and e N , are formed making use of the periodic 
nature of r(i): n e.g., because x N . +2 = $2 we have e N , = x N , — x 2 . 
Finally, only the magnitudes are retained, and these are rank ordered. 



* Upper and lower case letters indicate Fourier Transform pairs. 
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The result is the vector e: 

• - («i ,«.,-•• , e x .), e y ^ e 2 ^ ■ ■ ■ ^ e N . ^ 0. (14) 

(Any e k = could be discarded.) 

3.2 Formation of the Pdj for Binary Transmission 

The number of levels N is normally a power of two. In such cases 
the pdf p,(") for N levels can be built up from the pdf for the binary 
case as discussed below. Thus the binary case is of fundamental im- 
portance. Ignoring the Gaussian noise for the moment, the error voltage 
z of equation (fi) is 

N. 

z = X) a * e *» (15) 

where the e k are elements of e and for the binary case each a k takes 
on the values ±V with equal probability. There are 2 A * possible realiza- 
tions of the sequence (a t , a 2 , • • • , a^,), each occuring with probability 
2~ N '. Each realization yields a value for z, (not necessarily all different), 
which contributes a delta function of weight 2~ ' to the pdf of z. Since 
N , can be 100 or more, an attempt to form z for each possible sequence 
of a k would be futile (2 I()0 = 10 30 ). Instead, decomposition and quan- 
tization are used to reduce this formidable task to a sequence of rather 
simple ones. 

3.2.1 Decomposition 

The solution used here to circumvent the overwhelming computa- 
tional problem is to decompose the N, samples into blocks of K samples, 
and to find the pdf of ISI due to the samples for each block.* The 
resulting pdf's are convolved together to form the final binary pdf 
of z. Using specific numbers for concreteness, if N, = 54 and one chooses 
K = 9, then z can be written as a sum of six random variables 
Ai , • • • , A R , where 

A, : = 2Z a &> i= !. •" i 6 - ( 16 ) 

*=9i-8 

Each A, ^ is a discrete random variable, being the sum of nine discrete 
random variables. As the pdf of each A t is formed, it is convolved 
with previous ones until all 54 samples have been included. 



* Assume for the moment that 2V, is an integer multiple of K. A rule-of-thumb 
for discarding some of the small error samples in order to reduce the computation 
time is described in the following text. 
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Consider the formation of A, . Since a k can take on values ±V, 
there are 2 9 possible values for A x . The largest value, denoted as A* lt 
occurs when all a k - V, and has value At = l 7 (e, + e 2 + ■ ■ ■ + e ). 
The 2 9 delta functions that make up the exact pdf of A , will therefore 
lie in the interval [ — A* x , Af]. Since the pdf is symmetrical it is sufficient 
to compute it for nonnegative values of A t only. Every combination 
of ±V values for a t , • • • , a 9 is tried. For each the corresponding value 
of A t is found according to equation (16), and is discarded if A, < 0. 

The following simple technique insures that all combinations are 
included. A 9-digit binary number B is initialized to zero, and then 
in turn incremented by one until all digits are one (2 9 iterations in all). 
For each value of B, the value of a k is V if the fcth digit of B is 1, and 
— V otherwise. 

3.2.2 Quantization 

Decomposition alone would not significantly reduce the size of the 
computation, since the total number of delta functions in the final 
binary pdf of z would still be on the order of 2 54 . Quantization of the 
voltage axis is used to keep the number of possible levels of z within 
reasonable bounds. 10 

The interval [0, A* t ] is marked off with 2N, + 1 "location" points 
ti , i = 0, 1, • ■ ■ , 2iV, . Associated with each t { is a probability p, . 
Instead of recording the exact value of Ax , it is tested to see which t t 
it is nearest, and the corresponding p { is incremented by 2 -9 . There 
are many roughly equivalent ways of assigning the values t t . The 
following scheme was adopted for its computational simplicity. The 
interval [0, A^] is broken into two parts at some level kA* x , and then 
each part is uniformly quantized into N x levels. Hence, the t t are 
given by 

t = IkAV/Nt, i = 0, 1, •■• ,N l , (1?) 



ObA? + (1 - k)A* 1 (i/N l - 1), i = N,-\- 1, ■■■ , Ml 

For k = 1/2, the quantizing is therefore uniform over the whole 
interval. For k > 1/2 the quantizing is finer for larger values of Ai , 
so that the most important levels of error voltage are most accurately 
represented. An efficient choice for k is 0.6, as shown in Section V. 
(However, uniform quantizing may have other advantages since con- 
volution could be performed using a Fast Fourier Transform 
algorithm. 11 

The pdf 's of A 2 , ■ ■ ■ , A 9 are computed in the same way. After 
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each has been found, it is convolved with the resultant pdf for the 
previous ones. Quantization is also used in the convolution process. 

3.3 Formation oj the N-Level Pdf 

Once the binary pdf for z has been obtained, it is a straightforward 
matter to find the pdf for 4-, 8-, 16-, • • • level sources. The key to this 
is to envision N level transmission as the sum of log2 N independent 
binary transmission systems. 10 For instance, an 8-level source with 
voltage range from -F to F can be decomposed into three binary 
sources with symbols f k , g k , and h k having only values ±F. Each 
8-level symbol a k can be written as: 

o. = ih + 2 /' + ** (18) 

and the error voltage for eight levels can be written in the noise-free 
case as 

z = 7 2 Ue k + f £ 9ke k ItE h k e k 
= $E l + *E % + \B U (19) 

where E\ , E 2 , and E 3 are independent random variables, all having 
the same pdf as z in the binary case. (The generalization to any power 
of 2 is straightforward.) Since z is the sum of three random variables, 
and the pdf of each is known, it is a simple matter to convolve scaled 
versions of the binary pdf to obtain the 8-level pdf. Quantization is 
used here also, employing the scheme described above. The final pdf is 
thus approximated over the interval [0, D-V] by a set of 2JV\ + 1 
delta-functions having locations U similar to those in equation (17), 
and probabilities p.,- . By symmetry there is the same pattern of delta- 
functions at location — U with probabilities pc . 

3.4 Calculation oj the Error Probability 

The Gaussian random variable n of equation (6) is reinserted by 
convolving its pdf with that found above. The result is simply 

P,(x) =4-Ip.|e- (l -' OV2 "+e- (I+ "' ,w }. (20) 

V -7T0- «'-0 

From equation (8) the error probability is then 



P. = 2(1 - AT*) ^ p, 



Q[Z^) + Q (V + ti 



tr(N - 1)/ ' * \cx(N - l)/_ 



(21) 
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where Q (x) is the normal probability integral 

Q(x) = -^= f\-" /2 dt. (22) 

The accuracy of the method is examined in Section V, following a 
set of examples of the method applied to typical channel distortions. 

3.5 Reducing the Computation Time by Discarding Samples 

In the preceding discussion all A r s error samples of equation (14) 
were involved in the computation of the pdf of ISI. In most practical 
cases, however, a sizable number of the error samples are very small, 
and consequently their effect on the nature of the pdf as calculated 
is negligible. In order to save some computation time it behooves one 
to truncate the rank-ordered vector e of (14) at some index M, and to 
lump the other samples together in some fashion. This is an optional 
procedure, since the computation time grows only approximately 
linearly with the number of blocks of K samples processed, but it can 
indeed improve the efficiency of the method. 

One way to approximate the effect of the remaining samples is to 
treat the random variable 

A r = £ a k e k (23) 

as a Gaussian random variable with variance 4 

a\ = E[a 2 } £ el (24) 

The effect of these samples is thus a contribution to the Gaussian 
noise n, replacing the previous variance a 2 in equation (21) with a\ = 

2 I 2 

a + a T . 

The question of choosing M still remains. One rule of thumb is that 
the sum of the samples lumped together should not exceed 1 percent of 
the total sum D. This was found to be very useful empirically: for all 
choices of channel studied P e was the same whether all N s samples 
were used, or only M of them, while if only M were used the saving 
in computation time was significant. 

There are some pathological cases of mainly academic interest where 
D is theoretically infinite. This is true, for instance, for an ideal band- 
limited signal with a timing error. Saltzberg has shown, using an 
upper bound method, that the error probability can still be small in 



DIGITAL SYSTEM ERROR PROBABILITY 2065 

such cases. 4 The present method will not work, however, since the 
value of D is a crucial landmark in the computational procedure, and 
it must be finite. Such a restriction does not significantly reduce the 
power of the method. 

IV. EXAMPLES OF ERROR RATE CALCULATION 

In each of the examples the following definitions are used: 

p 

P e is normalized by the number of information bits in each symbol 
so that comparisons between different values of N will be more mean- 
ingful. Signal-to-noise ratio is given by the ratio of the variance 
E[c 2 ] of transmitted levels to average noise power. Since twice as 
much power is transmitted (if V is unchanged) when partial response 
IV coding is used, h 8 = 1 for uncoded sources, and h 8 = 2 for partial 
response IV sources. 

(i) Error in Sampling Time: 

A convenient example of an ideal signal shaping characteristic 
S(f) is shown in Fig. 2. S(f) satisfies the Nyquist criterion 
and consequently if R(j) of Fig. 1 were equal to S(f) no inter- 
symbol interference would be present at the receiver. Instead 
it is assumed that the sampling time is in error by pT seconds, 
so that R(f) is given by 

R(1) = S(/)e~' 2 " pr . (26) 

Figure 3 shows the vector of error samples e of equation (14) 



S(t) = slnc( r t/T) slnc(t/T) 
y =o.n 



1 1 + y 

2T 2 T 
FREQUENCY, f — * 

Fig. 2 — Ideal signal shaping characteristic. 
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Fig. 3 — Error samples e* for timing error. 



for p = 0.02 and p = 0.1, and for coded and uncoded sources. 
The samples fall off rapidly in each case, and the significant 
samples are somewhat larger for partial response IV coding. 
In order to include 99 percent of D, six blocks of nine samples 
must be used in the partial response IV case, and eight blocks 
of nine in the uncoded case. Figure 4 shows the resulting prob- 
ability density functions for an uncoded source with timing 
error p = 0.1. (In the quantization process N, = 40 so that 
81 locations were used in the interval [0, V-D].) The binary and 
8-level pdf's are shown along with a Gaussian pdf having the 
same value of MSE. The Gaussian function is a reasonably 
good approximation to the binary case near the origin, but 
is of course very poor near z = DV, since the pdf's of inter- 
symbol interference are strictly zero beyond this point. Figures 5 
and 6 show curves of error-rate versus SNR for several values 
of timing error. An 8-level source is of course much more sen- 
sitive to timing error than is a binary source. In the absence 
of intersymbol interference a partial response IV signal requires 
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about 3 dB more signal-to-noise ratio than an uncoded signal 
to achieve the same error-rate, under a constraint of equal 
average signal power. It is clear from the figures that this 
"SNR cost" need not be maintained when intersymbol inter- 
ference is present. The partial response IV signal is more sen- 
sitive to timing error than is the uncoded signal, such that at 
p = 0.15 the cost is around 9 dB. However, for types of dis- 
tortion other than timing error, this situation can be markedly 
reversed, since a partial response IV signal is rather impervious 
to distortions near dc or the Nyquist frequency. 1 



(ii) Single Echo in the Channel: 

Echoes frequently occur in transmission channels, generated for 
instance by mismatched terminations. A sizable amount of 
analysis has been done on echo theory in relation to system 
distortion. 1 ' 1 " Here a single echo of amplitude g and relative 
delay /37 1 is assumed to occur in the ideal channel of Fig. 2, 
so that 



R(D = S(/)(i + ge 



-i2rfPT 



)• 



(27) 



0.02 



0.005 




Fig. 4 — PDF of intersymbol interference. 
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Fig. 5 — Error rate vs SNR for timing error. 



Because s(t) is symmetrical and introduces no intersymbol 
interference, an echo at /3T is identical in its effect to an echo at 
(± /3 ± n)T for any integer n. For instance, echoes at = 
± 0.4, ± 3.4 ± 0.6 all have identical error sample vectors. 
Consequently one need only consider values of (3 between 
and 0.5. 

Figure 7 shows curves of error rate versus SNR for echoes 
of various strengths, and for an 8-level partial response IV 
coded source. Direct calculation shows that, because of the 
narrow excess bandwidth y of £(/) in Fig. 2, the sample error 
variance MSE of equation (10) is essentially independent of 
echo position. Echo strengths were chosen to yield convenient 
values of MSE = 0, 0.001, 0.003, and 0.005. It is clear that 
echo location has a strong effect on the error rate curves. Echoes 
at /3 = .5 yield the largest error rates. The error rate does not 
necessarily vanish as SNR -* °o , as indicated by the asymptote 
labelled "isi alone." 
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(in) Equal Error Samples: 

Consider the error sample vector e consisting of n identical 
elements, e k = u, i = 1, • • • , n. A considerable amount of 
insight is gained by examining this case, although it is unlikely 
that any physical system would give rise to such a vector of 
error samples. In addition, one can derive the pdf analytically 
for the binary case providing a useful check on the computer 
method. 

The error voltage z of equation (15) clearly has the value 
uV(n — 2r) if and only if r of the symbols a k equal —V, and 
the rest equal V. Such an event occurs with probability (") 2~" 
so that z has the pdf 



V: 



M = £H2-"«(e-iiV(n-2r)) > 



(28) 



and the error probability follows immediately using the method 
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Fig. 6 — Error rate vs SNR for timing error. 
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36 40 

SNR in dB 



Fig. 7 — Error rate va SNR for single echo. 



of Section 3.4: 



P> = £ ( n ) 2""Q[ VSNR [1 - u(n - 2r)]]. 



(29) 



The case n = 1 is the same as an echo positioned at /37 1 = 0. 
Figure 8 shows curves of error rate versus SNR as a function 
of n. (Curves formed using equation (29) and the method just 
given were indistinguishable.) The samples have size u = 
0.5/w so that d = 0.5 for all n, while MSE = 0.25/n. From 
the curves it is clear that a few large error samples are more 
degrading than many small ones for the same value of D. This 
is intuitively reasonable since only very special source sequences 
\a k \ will cause the error voltage z to be large when e consists 
of many small error samples, and such sequences occur with 
very small probability. Figure 9 shows similar curves, where 
now the samples have sizes 0.1/ y/n or 0.2/ y/n, so that MSE 
is constant at 0.01 or 0.04. For MSE = 0.01 error rate is some- 
what insensitive to the number of error samples, because the 
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increasing value of D = 0.1 Vn with n is offset by the decreasing 
probability that sequences \a k \ yielding large errors will occur. 
However, as n increases for MSE = 0.04, D approaches 1, the 
value at which errors occur due to intersymbol interference 
alone, (D = 1 for n = 25). The offset in probability is not 
strong enough here, and error rate is very sensitive to n. 

V. ACCURACY OF THE METHOD AND THE SALTZBERG BOUND 

The applications of quantization at various steps in the computa- 
tion shift the locations of some of the delta-functions slightly in one 
direction or the other, introducing some error in the t t locations of 
equation (17). It would be extremely difficult to estimate analytically 
the error resulting in the value of P e . Instead an empirical check was 
made over several channel characteristics. Figure 10 shows how error 
rate converges with increasing Ni for a typical example of channel 
distortion. Convergence was typically most rapid for k = 0.6. For 
high values of SNR uniform quantization (k = 0.5) was noticeably 
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Fig. 8 — Error rate for equal error samples. 
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Fig. 9 — Error rate for equal error samples. 

inferior. Because computation time grows rapidly with Ni , values of 
A7i = 40 and k = 0.6 were used in the examples of Section IV. With 
these values the algorithm is acceptably rapid, even on time-sharing 
facilities. 

A much faster and simpler scheme was proposed by Saltzberg. 4 
This technique found an upper bound for the error probability, but it 
was not clear from the theoiy how tight a bound was being computed. 
Figures 11 and 12 compare the error rate using the method given 
above with the upper bound found using this technique. For binary 
transmission the bound is between one and three orders of magnitude 
above the true value. For 8-level transmission the bound is less tight. 
The discontinuities in the bound versus signal-to-noise ratio occur 
due to the optimum partitioning of the error samples into two sets in 
the upper-bound method. 

(i) Choice oj the Block Size in the Method: 

Given M error samples (the M "significant" samples from the 
original set of N,), it was desired to find the binary pdf of z in 
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equation (15). To do this, the M samples were partitioned into 
M/K blocks of K samples each. Consequently one must compute 
M/K pdf's based on sets of K samples, and perform M/K — 1 
convolutions of these pdf's to obtain the binary pdf of z. The 
speed of the convolution process depends on 2iV\ + 1, the number 
of quantization levels. By direct examination of the algorithm 
one finds: (i) that to form a pdf trying all combinations of K 
samples requires 2 K multiplications; (ii) that each convolution 
requires 5(2iVi + l) 2 multiplications. The total number of 
multiplications is therefore 



N n 



= 2 K M/K + (M/K - l)5(2N 1 + l) 2 . 



(30) 



For example, if N t = 40 and M = 50, a search shows that Af muU . 
has a minimum with respect to K of 1.2 X 10 5 at K = 12. The 
saving in computation effort is dramatic: if no decomposition 
were used (K = M), iV mult . would be 1.1 X 10 15 . More generally, 
examination of equation (30) reveals that if M/K 2> 1 then 
Nmuit. varies linearly with M. Therefore Af mu i t . has a minimum 
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Fig. 10 — Convergence of error-rate values. 
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Fig. 11 — Comparison with upper bound. 

with respect to K that depends only on N x . This minimum is 
K = 12 for JV, = 40, and K = 13 for JVi = 60, although N M . 
is rather insensitive to K for Ke (8, 15). 



VI. CONCLUSIONS 

A method has been developed that accurately computes the prob- 
ability of error for a multilevel digital signal contaminated by intersym- 
bol interference and noise. The method constructs a close approximation 
to the probability density function for the intersymbol interference, 
which can provide useful insight into the nature of the interference. 
The method is rapid enough for use on time-sharing facilities, and can 
provide a useful tool to aid the analysis and design of digital communi- 
cation systems. 

Starting with the transfer function or impulse response of the 
channel, a set of error samples is found, and the method operates on 
these to form the probability density function of intersymbol inter- 
ference. Partial response coding of the source is easily included if 
desired. The formidable size of the computation that has heretofore 
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blocked direct calculation of this probability density function is cir- 
cumvented by partitioning the problem into a sequence of easily 
handled computations. Quantization keeps the total number of ele- 
ments in the probability density function within reasonable limits, 
and the density function for binary transmission is used repeatedly 
to form the density function for 4-, 8-, 16-, • • • level transmission. 
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APPENDIX 

Extension to Other Coding Types 

The method described above considers only two coding types: the 
uncoded and partial response IV cases. With slight adjustments in the 
method one can also treat other interesting cases. The class of coders 
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considered is that of Kretzmer 9 - 13 : each symbol c k is related to the 
input symbols {a k } by 

c k = b a k + M*-i + b 2 a k - 2 + • • • + b v a k - v , (31) 

where the elements 6, of the vector b are integers. (The uncoded case 
is then b n = 1, all other 6, = 0). Kretzmer tabulates five partial response 
classes which have proven most interesting: (I) b = (1, 1); (II) b = 
(1, 2, 1); (III) b = (2, 1, -1); (IV) b = (1, 0, -1); and (V) b = 
(—1, 0, 2, 0, —1). Other simple types exist as well, such as b = 
(1, 0, 0, 0, 0, 0, 0, 0, 0, —1), a generalization of class IV partial response 
that has five "lobes" in the signal spectrum. 9 

To see how codes of the type in equation (31) could be included in 
the method, note that there are only three places where the coding 
scheme used makes a difference: (i) in forming the coded error samples, 
as in equation (7), (it) in finding P e from the pdf, as in equation (8), 
and (Hi) in evaluating signal-to-noise ratio, as in equation (25). 

(i) The coded error samples are easily found for any vector b. 
Using equation (30) in equation (4), and manipulating 

z = X] 2 b m a k - m x k + n 

k m = 

= Z & »Z ajX i+m + n 

m = | 



where 



>,• = £ b m x i+m (32) 



are the coded error samples. 
(it) Once the pdf of isi has been found from the coded error samples, 
the error probability follows as in equation (8). To use signal 
power efficiently a coder will be chosen so that the transmitted 
voltage levels are equally spaced. Thus one need only evaluate 
the probability of an outer level in order to adapt equation (8) 
to this larger class of coders. An outer level of c k occurs only 
if all relevant a k have their maximum size and correct polarity. 
For AMevel a k symbols then, if M of the coefficients b { in equa- 
tion (31) are nonzero, each outer level has probability N~' w . 
Thus M replaces h in equation (8). 
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(Hi) Finally, since the symbols a k are statistically independent and 
have mean square value E[a 2 ] as in equation (25), it is a simple 
matter to show that the symbols c k have mean square value 
E[c 2 ] = h, E[a 2 \, where 

K - £ & . (32) 

m-U 

Therefore h, replaces h, in equation (25) in the evaluation of 
signal-to-noise ratio. 
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